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ABSTRACT 

This study examines ' the use of question prompts and peer 
interactions as scaffolding strategies to help undergraduate students with 
their problem-solving processes on an ill-structured task. The mixed research 
method, combining both experimental and comparative multiple-case studies, 
was used to study the outcomes as well as the processes of students’ problem- 
solving activities in terms of problem representation, developing solutions, 
constructing argumentation, and monitoring and evaluation. The result of the 
experimental study showed that the students working with peers and also 
receiving question prompts (PQ) significantly outperformed the other 
treatment groups, that is, individuals with question prompts (IQ), 
individuals without question prompts (IC) , and peers without question prompts 
(PC) . At the same time, though the students in the IQ group did less well 
than the PQ group in the process of problem representation, they 
significantly outperformed the PC and the IC groups in the processes of 
problem representation, justifications, and monitoring and evaluation. There 
were no significant differences between the PC and the IC groups in any of 
the four problem-solving processes. It appeared that question prompts were a 
superior scaffolding strategy over peer interactions in supporting students’ 
ill-structured problem-solving processes. However, the comparative, multiple 
case studies revealed the complexity of. the peer interaction context and the 
relationship between question prompts and peer interactions. While this study 
confirms previous findings on the effectiveness of question prompts in 
facilitating students’ cognition and metacognition, it also indicates the 
benefits of peer interactions, which were contingent upon group members’ 
active and productive engagement, that is, questioning, explaining, 
elaborating and providing feedback among peers. The study implies that, in 
order for students to gain full benefits from peer interactions, the peer 
interaction process itself needs to be scaffolded, especially when students 
were novice problem solvers; and question prompts, through expert modeling, 
may serve to facilitate this process. (Contains 23 references.) (Author/AEF) 
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Abstract 

This Study examines the use of question prompts and peer interactions as scaffolding strategies to help undergraduate 
students with their problem-solving processes on an ill-structured task. The mixed research method, combining both 
experimental and comparative multiple-case studies, were used to study the outcomes as well as the processes of students' 
problem-solving activities in terms of problem representation, developing solutions, constructing argumentation, and monitoring 
and evaluation. The result of the experimental study showed that the students working with peers and also receiving question 
prompts (PQ) significantly outperformed the other treatment groups, that is, individuals with question prompts (IQ), individuals 
without question prompts (IC), and peers without question prompts (PC). At the same time, though the students in the IQ group 
did less well than the PQ group in the process of problem representation, they significantly outperformed the PC and the IC 
groups in the processes of problem representation, justifications, and monitoring and evaluation. There were no significant 
differences between the PC and the IC groups in any of the four problem-solving processes. It appeared that question prompts 
were a superior scaffolding strategy over peer interactions in supporting students’ ill-structured problem-solving processes. 
However, the comparative, multiple case studies revealed the complexity of the peer interaction context and the relationship 
between question prompts and peer interactions. While this study confirms previous findings on the effectiveness of question 
prompts in facilitating students’ cognition and metacognition, it also indicates the benefits of peer interactions, which were 
contingent upon group members ’ active and productive engagement, that is, questioning, explaining, elaborating and providing 
feedback among peers. The study implies that, in order for students to gain full benefits from peer interactions, the peer 
interaction process itself need to be scaffolded, especially when students were novice problem solvers; and question prompts, 
through expert modeling, may serve to facilitate this process. 

Problem Statement 

Complex, real-world problem solving is an essential component of learning. Based on previous research (e. g., Bransford, 
Brown, & Cocking, 2000; Bransford & Stein, 1993; Jonassen, 1997), engaging students in complex, ill-structured problem- 
solving tasks not only helps them to apply knowledge in real-world situations, but also to facilitate knowledge transfer. 
However, previous research has also pointed to students' deficiencies in problem-solving skills, for instance, failing to apply 
knowledge learned in one context to another, especially when solving problems on ill-structured tasks (Gick and Holyoak, 1980; 
Gick, 1986). While students' difficulties in problem solving are partly attributed to misconceptions or shallow conceptions of 
domain knowledge (P. J. Feltovich, Spiro, Coulson, & J. Feltovich, 1996), they are, to a greater extent, due to a lack of 
metacognitive knowledge (Brown, 1 987). 

Therefore, it follows that supports should be provided to students during problem solving in cognition and metacognition 
through various scaffolding strategies, such as coaching through prompts (Scardamalia, Bereiter, & Steinbach, 1984; 
Scardamalia, Bereiter, McLean, Swallow, & Woodruff, 1989; Schoenfeld, 1985; King, 1992), modeling through reciprocal 
teaching or peer-regulated learning (e.g., Palincsar & Brown, 1984; Palincsar, Brown, & Martin, 1987), and guiding students to 
self-generate questions (King, 1991). These strategies were found to be effective in fostering comprehension, monitoring 
cognitive thinking, facilitating general problem solving (e.g., Palincsar & Brown, 1984; Scardamalia et ah, 1989), and supporting 
reflective thinking (Lin, Hmelo, Kinzer, & Secules, 1999). 

Purpose of the Study 

The purpose of the study was to investigate the effects of question prompts and peer interactions in scaffolding undergraduate 
students' problem-solving processes on an ill-structured task. Although these two strategies had been studied in previous 
research, few studies had been conducted on their use to support students’ ill-structured problem solving. Hence, this study was 
focused on the effects of question prompts and peer interactions in scaffolding students' problem-solving processes on an ill- 
structured task, especially in problem representation^ solution^ justifications, and monitoring and evaluation, which characterize 
the major processes of ill-structured problem solving according to previous research (e.g., Sinnott, 1989; Voss, 1988; Voss and 
Post, 1988). It is hoped that the findings of this research can be applied in computer-based and web-based instructional design, in 
the context of both distance education and classroom instruction. 
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This study specifically addressed the following questions: 

Question 1. Does the use of question prompts have an effect on students’ problem solving on an ill-structured task in problem 
representation, solution development, justification, and monitoring and evaluation of solutions? 

Question 2. Does the use of peer interaction have an effect on students’ problem solving on an ill-structured task in problem 
representation solution development, justification, and monitoring and evaluation of solutions? 

Question 3. Does the use of question prompts combined with peer interaction have an effect on students’ problem solving on 
an ill-structured task in problem representation, solution development, justification, and monitoring and evaluation of solutions? 

Question 4. How does the use of question prompts influence students' cognition and metacognition in the process of 
developing solutions to ill-structured problems? 

Question 5. How does the use of peer interactions influence students’ cognition and metacognition in the process of 
developing solutions to ill-structured problems? 

Operational Definitions 

In this study, question prompts refer to a set of static questions, both content-specific and metacognitive types, which were 
generated by the content domain experts and were designed to facilitate cognition and metacognition and guide students through 
problem-solving processes. They were related to a problem-solving task which students were engaged in. For example, “What 
are the parts of the problem?” was intended to provide cues to students, activate their prior-knowledge, and lead them to represent 
the problem. When prompted to reflect on their solutions, students were asked “How do I justify this specific system design? If I 
develop a web-based solution, for example, can I explain why I took that approach?” The question prompts were delivered 
through the web, which students had access to while working in a computer laboratory. 

Peer interactions are defined as verbal interactions of students working together in small groups of three or four to engage in 
a task of ill-structured problem solving. Students were expected to engage in a problem-solving task, actively interact with each 
other to negotiate meanings, to help each other construct meanings, and collaboratively develop solutions to a problem. 
Although they may have different abilities, skills and background experiences, they were not assigned specific roles. 



Method 

Participants 

1 15 undergraduate students were recruited from three class sections of an introductory course in Information Sciences and 
Technology (1ST) at a major university in the United States to participate in the experimental study, and 19 of them also 
participated in the comparative, multiple-case studies. The course was designed not only to introduce basic concepts and provide 
an overview of information sciences and technology, but also to incorporate collaborative learning and problem-solving 
experiences. It consisted of two lectures and one laboratory session each week. The 75-minute lecture session was held by a 
professor. The 1 15-minute laboratory session was conducted by a teaching assistant. The primary purpose of the lab was to 
provide hands-on experience in information sciences and technology and develop technology skills as well as problem-solving 
and collaboration skills. There were two teaching assistants attached to the three class sections, with the principal investigator 
being one of them. All the three class sections shared a common curriculum and a core textbook, and the three professors and the 
two teaching assistants were considered equivalent in terms of their expertise and teaching experience. Due to the relative large 
size of the class (about 50 students in each class section), the class web site was used as a supplementary delivery medium to 
foster classroom instruction and monitor laboratory activities. The students were often required to work in teams to complete a 
course project or laboratory tasks. 

Design 

A mixed study design, combining an experimental study with comparative, multiple-case studies, was applied. According to 
Greene, Caracelli, and Graham (1989), mixed study methods help a researcher to seek triangulation of the results from different 
data sources, examine overlapping and different facets of a phenomenon, discover paradoxes, contradictions, and fresh 
perspectives, and expand the scope and breadth of a study. The experimental study, designed to answer research questions 1-3, 
was conducted to measure the students' problem-solving outcomes on an ill-structured task in the four problem-solving processes: 
problem representation, solutions, justifications, and monitoring and evaluation, and in four different treatment conditions: 
individuals with question prompts (IQ), individuals without question prompts (IC), peers with question prompts (PQ), and peers 
without question prompts (PC). The comparative, multiple-case studies, through observation, hterviews, and think-aloud 
protocols, were carried out to gain insights into students' problem-solving processes, especially their cognition and 
metacognition, as influenced by question prompts or peer interactions. The case studies were expected to seek explanations to 
research questions 4-5. 

The Quasi-Experimental Study 

As the experimental study was integrated into the curriculum and carried out in a natural classroom setting, the participants 
were assigned to different treatment groups as intact groups. From Questions 1-3 the following hypotheses were generated: 

1 . Students working individually and also receiving question prompts will demonstrate better problem-solving skills on an 
ill-structured task than their counterparts who did not receive the question prompts in (a) problem representation, (b) 
developing solutions, (c) making justifications and (d) monitoring and evaluating solutions. 
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2. Students working with peers, with or without question prompts, will demonstrate better problem-solving skills on an 
ill-structured task than students working individually, with or without question prompts, in (a) problem representation, 
(b) developing solutions, (c) making justifications, and (d) monitoring and evaluating solutions. 

3. Students working with peers but also receiving question prompts will demonstrate better problem-solving skills on an 
ill-structured task than all the other treatment groups (PC, IQ, and IC) in (a) problem representation, (b) developing 
solutions, (c) making justifications, and (d) monitoring and evaluating solutions. 

The participants of the four different conditions were given the same task to solve during a 1 15-minute laboratory session. 
The task presented an authentic scenario (Table 1) concerning the content domain of 1ST, which required students to develop a 
solution report to the problem presented. 

Tabl e L The ill-structured problem -solving task 

Many customers complain that they have difficulty finding items in a large store. This problem especially affects 
college students, who often have very little time for shopping. Since students are major customers in this small 
college town, the manager of a local major store has hired you (or your team) as a consultant to propose IT-based 
solutions to the problem. Your task is to make suggestions about the features to be included in a new information 
system. As part of this, you are to develop a simple model illustrating your proposed system. Based on the findings 
of a survey, the proposed information system should be able to help customers fnd items quickly, to present an 
overall view of all the items on a shelf and an aisle, and to map out the shortest route for getting all the items a 
customer needs to purchase. There may be some other important factors you may need to consider. 



The participants who were assigned to the Question-Prompt conditions (PQ and IQ) received question prompts at the same 
time as they received the problem-solving task. The question prompts consisted of 10 question prompts categorized into four 
types of prompts: 

a) Problem Representation Prompts: What is the problem? 

b) Problem Solution Prompts: How do I solve the problem? 

c) Justification Prompts: What are the reasons for,.. ? 

d) Monitoring and Evaluation Prompts: Ami on the right track? 

There are a number of questions in each category of question prompts, as illustrated in Table 2: 

Table 2 The justification prompts 

What are the reasons for mv proposed solution? 

• How would I justijy this specific system design. For example, if I develop a web-based solution, can I 
explain why I took that approach? 

• Do I have evidence to support my solution (i.e., the specific IT system I have proposed)? What is the chain 

of my reasoning to support my solution? 



The problem-solving reports were evaluated based on a judgmental rubric system to measure the four problem-solving 
processes by the principal researcher and another two raters. The interrater consistency was reached to ensure the reliability of 
the evaluation. A multivariate analysis of variance (MANOVA) (Stevens, 1986) was employed to analyze the relationships 
between question prompts and the four problem-solving processes, and between peer interactions and the four problem-solving 
processes across the four different conditions. As the MANOVA result was statistically significant, the univariate (ANOVA) 
results were examined for each dependent variable. For the significant univariate results, post hoc comparisons were performed 
to identify where the differences resided. 

The Comparative, Multiple- Case Studies 

Eight cases were studied. Four individuals, with two from the IQ and two from the IC condition, were selected for think- 
aloud protocols, which were conducted when each of them was engaged in the problem-solving task. In addition, observations 
and interviews were conducted on four selected groups to gather data about their problem-solving processes, two groups from the 
PQ and two groups from the PC condition. The multiple cases were analyzed for the purpose of theoretical replication, which 
either (a) predicts similar results or (b) produces contrasting results but for predictable reasons (Yin, 1989). 

Miles and Huberman’s (1994) data analysis model, which involves three subprocesses -data reduction, data display and 
conclusion drawing and verification, was used to guide the qualitative data analysis. The data analysis primarily consisted of the 
following steps: reading and jotting marginal notes on the transcripts; identifying patterns and labeling concepts; organizing 
labeled concepts into a data display matrix, identifying themes and drawing conclusions. The focus of the analysis was on cross- 
case comparisons viewed from different dimensions: the four ill-structUred problem-solving processes and the effects of question 
prompts and peer interactions on cognitive thinking and metacognitive skills. 

Results 

The Quantitative Results 

The results of multivariate analysis of variance showed overall differences for the treatment effect and the four dependent 
variables of problem-solving processes. The MANOVA results were statistically significant (F = 4.025, p< .001). Further, the 
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results of the univariate ANOVA tests indicated that there were significant statistical differences in all the four dependent 
variables, with an F ratio of 20.43, 8.27, 1 1.26 and 7.21 respectively (p < .001). Table 3 is a summary of post hoc Scheffe mean 
comparison. It shows several statistical mean differences among the four treatment conditions in the four dependent variables. 
Table 4 summarizes the descriptive statistics for the dependent variables by treatment groups. It displays means and standard 
deviations of different treatment groups by the four dependent variables. As each dependent variable has a different subtotal of 
scaled points, percentage was used to create a common basis for comparison of means among the four dependent variables. 



Table 3 Summary of post hoc Scheffe comparison 



Dependent Variable 


Reoresentine 

Problem 


Develooine 

Solutions 


Making 

Justifications 


Monitoring and 
Evaluation 


Comparison Group 


Mean 

Difference (%) 


Mean 

Difference (%) 


Mean 

Difference (%) 


Mean 

Difference (%) 


Peer Question (PQ) 
vs. 

Peer Control (PC) 


35.9* 

(PQ > PC) 


20.3* 

(PQ > PC) 


33.7* 

(PQ > PC) 


35.6* 

(PQ > PC) 


Peer Question (PQ) 
vs. 

Individual Question (IQ) 


17.6* 

(PQ>IQ) 


11.8 


7.7 


1.5 


Peer Question (PQ) 
vs. 

Individual Control (IC) 


39.8* 

(PQ>IC) 


21.3* 

(PQ > IC) 


27.3* 

(PQ > IC) 


34.8* 

(PQ > IC) 


Peer Control (PC) 
vs. 

Individual Question (IQ) 


-18.3* 

(PC < IQ) 


-8.5 


-25.9* 
(PC < IQ) 


-34.0* 

(PC < IQ) 


Peer Control (PC) 
vs. 

Individual Control (IC) 


3.9 


1.0 


-6.3 


-0.8 


Individual Question (IQ) 
vs. 

Individual Control (IC) 


22.2* 

(IQ>IC) 


9.5 


19.6* 

(IQ>IC) 


33.2* 

(IQ>IC) 



Note. 

a) The mean difference shown in this table is the subtraction of the second condition (on the lower line) from the first 
condition for example, 35.9 (Mean Difference for Problem Representation) = PQ - PC. 

b) Mean difference (%) is calculated using the values which appear in Table 4. 

c) (%). The mean difference is converted into percentage in order to create a common basis for mean comparison, as the 
subtotals for the four dependent variables are different. 

d) *. The mean difference is significant at the .05 level. 



Table 4 Descriptive statistics for each dependent variable by treatment group 



Dependent 

Variables 


Treatment Group 


Peer Question 

(PQ) 

(N=13) 

Mean% SD % 


Peer Control 
(PC) 

(N=ll) 

Mean % SD % 


Individual Question 

(IQ) 

(N=15) 

Mean % SD % 


Individual Control 
(IC) 

(N=I6) 

Mean % SD % 


Representing 

Problem 


62.3 17.4 


26.4 15.7 


44.7 15.5 


22.5 11.3 


Developing 

Solutions 


88.5 11.9 


68.2 17.1 


76.7 10.4 


67.2 11.1 


Making 

Justifications 


79.1 18.1 


45.5 12.5 


71.4 17.1 


51.8 18.7 


Monitoring and 
Evaluation 


61.5 31.1 


25.9 24.6 


60.0 30.2 


26.8 22.7 
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As it can be seen from Table 3 and Table 4, The statistical results generally confirm Hypothesis 7, showing that students 
working individually and also receiving question prompts (IQ) demonstrated higher problem-solving skills on an ill-structured 
task than the individuals who did not receive the question prompts (IC) in (a) problem representation, (c) making justifications, 
and (d) monitoring and evaluating solutions. However, the IQ group did not perform significantly better than IC group in (b) 
developing solutions. 

The statistical results only partially support Hypothesis 2. Students working with peers and also receiving question prompts 
(PQ) outperformed those working individually and without question prompts (IC) in all the four problem-solving processes. 
They also outperformed those working individually and also receiving question prompts (IQ) in (a) problem representation. 
However, no significant differences were found in any of the four problem-solving processes between the PC and IC groups. On 
the contrary, students working individually but receiving question prompts did significantly better than students working with 
peers but without question prompts in three processes: (a) problem representation, (c) making justifications, and (d) monitoring 
and evaluating solutions. 

In testing Hypothesis i, students working with peers and also receiving question prompts (PQ) demonstrated better problem- 
solving skills than the students in the other conditions (PC, IQ, and IC) in (a) problem representation. In (b) developing 
solutions, (c) making justifications, and (d) monitoring and evaluating solutions, students in the PQ condition did significantly 
better than those in the PC and the IC condition, though not than those in the IQ group. 

The Qualitative Findings 

The effect of question prompts. The cross-case qualitative analysis revealed that the question prompts supported students’ 
cognition and metacognition through directing attention to their problem-solving processes, articulating thoughts, and providing 
guidelines. First, question prompts served as a “reminder” to direct the students’ attention to some important information they 
might not have thought about. They helped students to represent the problem, make connections between different factors and 
constraints and link to the solutions. In addition, the question prompts also led the students to think about alternative solutions 
and the viability of their solutions. A group of students who failed to use the question prompts and thus failed to think about all 
the possibilities and alternative solutions indicated the important functions of question prompts. Second, it was observed that 
question prompts reminded the students to state their reasons and construct arguments for their proposed solutions. Third, 
students mentioned that the question prompts were useful to help them organize their thinking and break down the problem into 
small steps. Therefore, the question prompts may have served as expert modeling to support students' cognitive and 
metacognitive thinking by guiding problem representation, metacognition, and the justification process. 

The effect of peer interactions. The comparative case studies indicated that the greatest advantages of peer interactions lie in 
building upon each other’s ideas, questioning and providing feedback, providing multiple perspectives, and benefiting from 
distributed knowledge. Those attributes influenced students' cognitive thinking and metacognitive knowledge. It was observed 
in all the cases that when peers worked together, they typically started the problem-solving processes by brainstorming ideas, 
which were presented in the form of questions or suggestions, such as “How about...?” and “What do you think...?”. Then, an 
initial idea got further developed. It was also observed that during peer interactions, students asked questions, offered 
suggestions, elaborated thinking and provided feedback. Thus, peer interactions created an opportunity to ask, clariiy, explain, 
and elaborate. The great advantages of peer interactions, as the participants consistently pointed out in their interviews, were the 
multiple perspectives and different expertise different individuals brought to the problem-solving processes. 

Discussion 

Here is a summary of answers to research questions 1-5: 

1. Question prompts had a significantly positive effect overall on students' problem-solving processes on an ill-structured 
task, specifically in (a) problem representation, (c) making justifications and (d) monitoring and evaluating solutions. 

2. The use of peer interactions had a partially positive effect on students' problem solving processes on an ill-structured 
task in that, the students in the PQ condition significantly outperformed those in the IC condition in all the problem- 
solving processes and the IQ condition in problem representation; whereas the students in the PC condition did not 
perform significantly better than those in the IQ or the IC condition in any of the problem solving processes. 

3. In comparison with the separate use of question prompts and of peer interactions, the combination of question prompts 
with peer interactions showed the greatest positive effect overall on students' problem-solving processes on an ill- 
structured task. 

4. In the process of developing solutions to ill-structured problems, question prompts influenced students' cognition and 
metacognition by (a) directing attention, (b) articulating thoughts, and (c) providing guidelines for problem solving. 

5. In the process of developing solutions to ill-structured problems, peer interactions influenced students' cognition and 
metacognition by (a) building upon each other's ideas, (b) questioning and providing feedback, (c) providing multiple 
perspectives and (d) distributing cognition. 

The result of the experimental study showed that the students working with peers and question prompts (PQ) significantly 
outperformed the other treatment groups, especially the students without question prompts (either working individually or with 
peers), in all the four problem-solving processes. At the same time, the students working individually and with question prompts, 
though they did less well than the PQ group in problem representation, significantly outperformed the PC and IC groups in 
problem representation, justifications, and monitoring and evaluation. There were no significant differences between the PC and 
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the IC groups in any of the four problem-solving processes. It appeared that question prompts were a superior scaffolding 
strategy over peer interactions in supporting students’ problem solving on an ill-structured task. However, the comparative, 
multiple case studies revealed the complexity of the peer interaction context and the relationship between question prompts and 
peer interactions. While this study confirmed the findings cf previous research on the effectiveness of question prompts in 
facilitating students’ cognition and metacognition, it also showed the benefits of peer interactions, which were contingent upon 
group members’ active and productive engagement in peer interactions, that is, questioning, explaining, elaborating and 
providing feedback among peers. The findings supported Webb's (1989) research on the learning conditions for group 
collaboration. 

The study implies that, in order for students to gain full benefits from peer interactions, the peer interaction process itself 
need to be scaffolded, especially when students were novice learners in problem solving; and question prompts, through expert 
modeling, may serve to facilitate this process. Further research is suggested to examine the transfer effect of question prompts on 
students' self- generated questions if students are provided with similar question prompts over a period of time and if their 
improved skills in self- generated questions during problem solving will facilitate them to solve an ill-structured problem. More 
research efforts are also needed to examine group dynamics when investigating the role of peer interactions in scaffolding ill- 
structured problem solving. Group dynamics involve many aspects, including peer learning approaches, peer interaction patterns, 
students' perception and motivation about peer learning, any of which may have an impact on students' problem-solving 
performance. 
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